Recognition and localization of overlapping parts from sparse data in two and three dimensions

This paper discusses how sparse local measurements of positions and surface normals may be used to identify and locate overlapping objects. The objects are modeled as polyhedra (or polygons) having up to six degrces of freedom relative to the seneors. The approach operates by examining all hypotheses abont I,airings betvvxn sensed data and object surfaces and efficiently discarding inconsistent ones by using local constraints On: distances between faces, angles between face normals, and angles (relative to the surface normals) of sectors between sensed points. The described here is an extension of a method for recognition aItd localization of non-overlapping parts preViOwlY described in [Grimson & Lozano--P&rez 84) and [Gaston 2 LoZano-PCrez 841. 1. Problem Definition The specific problem we consider in this paper is how to identify a known object and locate it, relative to the sensor, using relatively few measurements. We want a recognition technique t,hat is applicable to a wide range of scnsors, so we make few assumptions about the sensory data availablc. We assume only that the sensory data can be proce~ed to obtain sparse measurements of the position and surface orientation of small planar patches of object surfaces in some coordinate frame defined relat,ive to the sensor. The measured positions are assumed to be within a known crror volume and measured surface orientations to be within a known error cone. Furthermore, the object is assumed to be overlapped by other nnknown object,s, so that, much of t,he data does not arise from the object of interest. If the object5 have only three degrees of freedom relative to the sensor ( two translational and one rotational), then the positions and surface normals need only be two-dimensional. If the objects have more than three degrees of freedom (np to three translational and three rotational), the position and orientation data must be three-dimensional. Since the measured data approximate small planar patches of the object’s surface, we assume that the objects can be modeled as sets of planar faces. Only the individual plane equations and dimensions of the model faces are used for recognition and localization. No face conncctivity information is used or assumed; the model faces do not wen have to be connected. This is important. Acknowledgments. This report describes research done at the Artificial Intelligence Laboratory of the Massachusetts Institute of Tc:chnology. Support for the Laboratory’s Artificial Intelligence rcsearch is provided in part by a grant from the Systcm 1)evelopment f?oundation, and in part by the Advanced Research Projects Agency nnder Ofice of Naval Research contracts N0001480-C-0505 and N00014-82-K-0334. I t is easy to build polyhedral approximations of moderately curved objects, but we cannot expect these approximations to bc perfectly stable under sensor variations. The connectivity among the faces is particularly vulnerable. Since our recognition method does not use face connectivity, but only local geometry, it can be readily applied to curved objects approximated by planar patches. Our basic approach to recognition proceeds in two steps: Generate Feasible Interprrtationa: A set of feasible interpretations of tilr senw data iu constructed. interpretations consist of pairings of rach sensed patch wilh some object surface on one of the known objccts. Interprclations inconsistent with local constraints, derived from the model, on the sense data are discarded. Model Test: The feasible interpretations are tested for consistency with surface equations obtained from the object models. An interpretation is legal if it is possible to solve for a rotation and translation that would place each senscd patch on an object surface. The sensed patch must lie inside the object face, not just on the surface defined by the equation. There are several possible methods of actually searching for consistent matches. For example, in Grimson and Lozano-PCrez [84] we chose to structure the seaich as the generation and exploration of an interpretation tree. That is, starting st a root nodk, we construct a tree in a depth first fashion, assigning mcawred patches to model faces. At the first level of the tree, we consider assigning the first measured patch to all possible. faces, at the next level, we assign the second measured patch to all possible face$, and 90 on. The number of possible interpretations in this tree, given a sensed patches and n surfaces, is n6. Therefore, it is not feasible to rxplore the entire search space in order to apply a model test t,o all possible interpretations. Moreover, since the computation of coordinate frame transformations tends to be expensive, we want to apply this part of the technique only as needed. The goal of the recognition algorithm is thus to exploit local geometric constraints to minimize the number of interpretations that necd testing, while keeping the computational cost of each constraint small. In the case or the interpretation tree, we need constraints betwern t,he data elements and the model elements that will allow us t,o remove entire subtrees from corlsideration without explicitly having to search those subtrees. In our case, we require that the distances and angles between a l l pairs of data elements be consistent with the distanccs and anglcs possible between their aysigned model elernents. The recognition algorithm described is related to several recent approaches to reccgnition based on geometric constraints [Rob and Cain 82, Bolles, Ilorand and Hannah 83, Faugeras and Hebert 53, Gaston and Lozano-Bkrez 84, Grimson and Lozano-1’L.rez 84, Stockman arld Estcva 841. See (Grimson and Lozano-Perez 841 for a more thorough discussion of the relevant literature. CH2152-7/85/0000/0061$01 .OO